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A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 . 1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 
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Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 
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2a)M This action is FINAL. 2b)D This action is non-final. - 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 
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5) D Claim(s) is/are allowed. 

6) S Claim(s) 1-17,26-30 and 32-44 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 
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10) S The drawing(s) filed on 19 October 2000 is/are: a)[x3 accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 185(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
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DETAILED ACTION 

Specification 

1 . Applicant is reminded of the proper content of an abstract of the disclosure. 

A patent abstract is a concise statement of the technical disclosure of the patent 
and should include that which is new in the art to which the invention pertains. If the 
patent is of a basic nature, the entire technical disclosure may be new in the art, and the 
abstract should be directed to the entire disclosure. If the patent is in the nature of an 
improvement in an old apparatus, process, product, or composition, the abstract should 
include the technical disclosure of the improvement. In certain patents, particularly 
those for compounds and compositions, wherein the process for making and/or the use 
thereof are not obvious, the abstract should set forth a process for making and/or use 
thereof. If the new technical disclosure involves modifications or alternatives, the 
abstract should mention by way of example the preferred modification or alternative. 

The abstract should not refer to purported merits or speculative applications of 
the invention and should not compare the invention with the prior art. 

Where applicable, the abstract should include the following: 

(1 ) if a machine or apparatus, its organization and operation; 

(2) if an article, its method of making; 

(3) if a chemical compound, its identity and use; 

(4) if a mixture, its ingredients; 

(5) if a process, the steps. 

Extensive mechanical and design details of apparatus should not be given. 

Response to Arguments 

2. Applicant's arguments filed 4/12/05 have been fully considered but they are not 
persuasive. 

3. In response to Applicant's arguments, filed 4/12/05, regarding claim 17, pages 
13-16, stating "When the local device does not recognize any of the keywords it sends 
the speech over a network to a remote device that has more extensive speech 
recognition capabilities." and that "White et al. do not disclose "searching for an 
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attention word ...and switching, upon finding the attention word, to a second context ..." 
The Examiner cannot concur with this assessment. 

While the applicant is permitted to define a particular word/phrase, "attention 
word" does not define a standard to an artisan. The Applicant argues an "attention 
word" functions, for example, to identify the user, to avoid false detections of requests 
and to distinguish between regular conversations and background noise." This is not 
claimed. As claimed, the "keywords" of White, appropriately define the limitations. 

White further teaches, "word spotting " searching (by scanning) for a keyword, 
(C.13.line 60-C.14.lines 10-his "wake up" command and then further proceeds to 
remote resources, C.6.lines 31-55-). The Applicant has stated "when the local device 
does not recognize any of the keywords...", page 14. However, the Examiner cannot 
concur. White "word spots", and in the least recognizes "wake up" (C. 13. line 60- 
C.14.line 10), searches for a keyword, and utilizes remote resources to perform the 
needs, that are associated with the keyword, upon finding the attention/keyword, as 
explained in the previous rejection's cited column and lines. 

In response to applicant's argument that Junqua, Giuliani, and White do not 
teach or suggest, "wherein the natural language interface abstracts each of the plurality 
of devices into a respective one of a plurality of grammars and a respective one of a 
plurality of lexica corresponding to each of the plurality of devices." The Examiner 
cannot concur. Junqua teaches having a plurality of devices coupled to the natural 
language interface module, Junqua fig. 1 , his natural language parser 26 and his digital 
tuner 40 and his recorder 44, C.2.lines 2-C.3.line 35-Junquas natural language interface 
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abstracts the plurality of devices, Fig. 2, C.3.lines 54-24-his semantic frame data 
structure... "whose domain is tuner commands"-is interpreted as his abstraction of 
devices, one of his digital tuner and recorder, his digital tuner, and recorder-The 
grammar is necessarily specific to each unit wherein the recorder is associated with 
specific recording grammar, and the tuner is associated with channel selection grammar 
as the natural language parser, 26, distinguishes between the two respective grammars 
and lexica, Fig 2-his domain specific knowledgebase, C.3.lines 60-C.4.lines 4-as 
explained, the natural language input by the speaker "Record me a comedy in which 
Mel Brooks stars and is shown before January 23 rd "-enters the "grammar" and 
commands specific to the recorder. 

In response to applicant's arguments, regarding claims 7 and 8, p. 20, searching 
for a natural language request upon receipt of a keyword. Applicant does not claim, 
"searching for a natural language request upon receipt of a keyword. 

Claim Rejections - 35 USC § 102 

4. . The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 

5. Claim 17 is rejected under 35 U.S.C. 102(e) as being anticipated by White et al 
(White, 6,408,272 b1) 
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As per claim 17, White et al teach a method of speech recognition comprising 
(figure 2): 

searching for an attention word based on a first context including a first set of 
grammar models (figure 2, his speech recognition engine 40, col. 6, lines 31-55, -the 
first set of grammar models resident upon the local device); and 

switching upon finding the attention word to a second context to search for an 
open- ended user request, wherein second context includes a second set of models, 
grammar and lexicons (col. 6, lines 45-55, col. 7, lines 2-16, col. 16, lines 1-15, col. 15, 
lines 14-20- White "word spots", searches for a keyword, and utilizes, switches to 
remote resources, including a second context, a second set of models and lexicons to 
perform the needs, that are associated with the keyword, upon finding the 
attention/keyword, also C.13.line 60-C.14.lines 10-his "wake up" command and then 
further proceeds to remote resources, C.6.lines 31-55). 

Claim Rejections - 35 USC § 103 

6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

7. Claims 1-16 and 26-64 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Junqua etal (Junqua, 6,324,512 B1) in view of Giuliani et al (Giuliani, 
Hands free Continuous Speech Recognition in Noisy Environment Using a Four 
Microphone Array) and White et al (White, 6,408,272 B1 ) 
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As per claims 1 and 5, Junqua et al teach a natural language interface control 
system for operating a plurality of devices comprising (figure 1 ): 

" feature extraction module coupled to the first microphone" this signal 
processing component 68, col. 15, lines 53-67); 

"a speech recognition module coupled to the feature extraction module, utilizes 
hidden Markov models; (His speech recognizer 20, col. 2, lines 35-55, Fig. 4); and 

"A device interface coupled to the natural interface module "(His natural 
language parser 26, col. 2, lines 52-61), "wherein the natural language interface module 
is for operating a plurality of devices coupled to the device interface based upon non- 
prompted, open- ended natural language request from a user" (his abstract, lines 1-5; 
col. 2, lines 62-67 his unified access controller 30, his digital tuner 40 and his recorder 
44, col. 3, lines 9-17). 

wherein the natural language interface module abstracts each of the plurality of 
devices into a respective one of the different grammars and a respective one of a 
plurality of lexica corresponding to each of the plurality of devices (Junqua fig. 1 , his 
natural language parser 26 and his digital tuner 40 and his recorder 44, C.2.lines 2- 
C. 3. line 35-Junquas natural language interface as cited in the previous rejection, 
abstracts the plurality of devices, his digital tuner, and recorder-The grammar is 
necessarily specific to each unit wherein the recorder has recording grammar, and the 
tuner has channel selection grammar as the natural language parser, 26, distinguishes 
between the two respective grammars and lexica, C.3.lines 60-C.4.lines 4-as explained, 
the natural language input by the speaker "Record me a comedy in which Mel Brooks 
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stars and is shown before January 23 rd "-enters the "grammar" and commands specific 
to the recorder). 

It is noted that Junqua et al teach the claimed invention but does not explicitly 
teach a 3 dimensional microphone array. However, this feature is well known in the art 
as evidenced by Giuliani et al who teach a four microphone array. Therefore, one of 
ordinary skill in the art at the time invention was made would have it obvious to 
substitute the microphone taught by Junqua by the array of microphone taught by 
Giuliani because it would improve the signal quality in a noisy environment (see Giuliani 
page 860). 

It is further noted that the combination teaches the claimed invention but does 
not explicitly teach wherein the speech recognition module can switch between different 
acoustic models and different grammars, wherein at least one of the different acoustic 
models and at least one of the different grammars is downloaded over a network. 
However, this feature is well in the art as evidenced by White et al who teach a 
distributed voice interface system that includes a remote system, which may 
communicate with a number of local devices where data can be downloaded from the 
remote system to the local devices at col. 3, lines 25-32 and col. 16, lines 1-15, and 
teaches having the natural language interface to the speech recognition, C.6.lines 35- 
40-resident on a VUI abstracting a plurality of devices, CAIines 55-60, C.5.lines 39-54, 
and C.6.lines 32-55-the natural language through the VUI functions to specific 
information grammars and lexica from remote locations to operate each of a plurality of 
local devices, thereby switching grammars, acoustic models). Therefore, one having 
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ordinary skill in the art at the time the invention was made would have it obvious to 
incorporate the combination as taught Junqua with Giuliani into a distributed system as 
taught by White et al because the data already present in each local device can be 
updated, replaces or supplemented ms desired to modify the voice user interface 
capability (White et al's col. 3, lines 28-34). 

As per claim 2, the combination teaches the plurality of devices coupled to the 
natural language interface module (Junqua figure 1, his natural language parser 26 and 
his digital tuner 40 and his recorder 44; White his speech recognition engine 40 and 70) 

As per claim 3, Junqua et al wherein the speech recognition module utilizes an 
N-gram grammar (col. 7, line 65 to col. 8, line 2). 

As per claims 4, Junqua et al wherein the natural language interface module 
utilizes a Probabilistic context free grammar ( figure 1 , his natural language parser 26, 
col. 5. lines 5-11). 

As per claims 6-8 (see rejection 1 above) the combination of Junqua with Giuliani 
and Junqua further teaches wherein the natural language interface abstracts each of 
the plurality of devices into a respective one of a plurality of grammar.. .plurality of 
devices" (Junqua fig. 1, his natural language parser 26 and his digital tuner 40 and his 
recorder 44, C.2.lines 2-C.3.line 35-Junquas natural language interface abstracts the 
plurality of devices, Fig. 2, C.3.lines 54-24-his semantic frame data structure... "whose 
domain is tuner commands"-is interpreted as his abstraction of devices, one of his 
digital tuner and recorder), his digital tuner, and recorder-The grammar is necessarily 
specific to each unit wherein the recorder is associated with specific recording grammar, 
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and the tuner is associated with channel selection grammar as the natural language 
parser, 26, distinguishes between the two respective grammars and lexica, Fig 2-his 
domain specific knowledgebase, C.3.lines 60-C.4. lines 4-as explained, the natural 
language input by the speaker "Record me a comedy in which Mel Brooks stars and is 
shown before January 23 rd "-enters the "grammar" and commands specific to the 
recorder, White's col. 4, lines 54 to col. 5, line 3, col. 6, lines 25-55 (each resident VUI 
may perform "word spotting" by scanning speech input for the occurrence of or more 
"keywords", and col. 7, lines 10-15, C.4.lines 54-C.6.line 55-White teaches having the 
natural language interface, C.6.lines 35-40-resident on a VUI abstracting a plurality of 
devices, CAIines 55-60, C.5.lines 39-54, and C.6.lines 32-55-the natural language 
through the VUI functions to specific information grammars and lexica from remote 
locations to operate each of a plurality of local devices, thereby White switches 
grammars, upon receipt of a keyword. Therein searching for the non-prompted 
open-ended, natural language requests upon the receipt and recognition of an "attention 
word" or keyword). 

As per claims 9 and 10, (see rejection of claim 1), the combination further 
teaches a grammar module for storing different grammars for each of the plurality of 
devices (Junqua, his digital tuner 40 and his recorder 44, col. 3, lines 9-17, The 
grammar is necessarily specific to each unit wherein the recorder is associated with 
specific recording grammar, and the tuner is associated with channel selection grammar 
as the natural language parser, 26, distinguishes between the two respective grammars 
and lexica, Fig 2-his domain specific knowledgebase, C.3.lines 60-C.4.lines 4, White 
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col. 5, lines 39-54, col. 6, lines 25-55, col. 7, lines 28-35, C.6.lines 35-40-resident on a 
VUI abstracting a plurality of devices, CAIines 55-60, C.5.lines 39-54, and C.6.lines 32- 
55-the natural language through the VUI functions to specific information grammars and 
lexica from remote locations to operate each of a plurality of local devices, White 
switches grammars, .... acoustic models from remote sites, upon receipt of a keyword. 
Therein searching for the non-prompted open-ended, natural language requests upon 
the receipt and recognition of an "attention word" or keyword). 

As per claims 11-16, the combination teaches wherein the device comprises a 
wireless device interface (White, col. 2, lines 55-64, col. 5, lines 39-47),. an external 
network coupled to the natural language interface (Junqua, his internet access 64); 
wherein said 3 dimensional microphone array includes the first microphone ( see 
Giuliani, his four microphone array) 

8. Claims 26-30, and 32-44 are the same in scope and content as claims 1 -1 6 
above and therefore are rejected under the same rationale. 

Conclusion 

9. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 

§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
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mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

1 0. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Lamont M. Spooner whose telephone number is 
571/272-7613. The examiner can normally be reached on 8:00 AM - 5:00 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Richemond Dorvil can be reached on 571/272-7602. The fax phone number 
for the organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-21 7-91 97 (toll-free). 
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